Device, system, and method for determining information relevant to a clinician

ABSTRACT

A system, method and device for determining and notifying a clinician of information relevant to the clinician. The method that is performed by the device or system includes identifying at least one keyword in a user profile of a clinician, identifying at least one content word in a new information item, determining a relevance score between the new information item and the clinician based on the at least one keyword and the at least one content word and when the relevance score is above a predetermined threshold value, generating a notification for the clinician indicating the new information item.

BACKGROUND INFORMATION

A clinician may provide healthcare or health-related information topatients in person or through communications such as onlinecommunications. Even with the knowledge that the clinician may beskilled in a concentrated medical field, the clinician may still referto external sources to aid in determining the proper healthcare orhealth-related information to provide to the patient. The clinician maybe limited (particularly in efficiency) in utilizing availableinformation.

Clinicians are interested in specific information and/or knowledge abouttheir own scientific domain or related specialties. With an everincreasing amount of available information (e.g., digital information),a clinician may find the process of manually tracking all availableinformation sources and manually mining these sources to retrievedesired information, particularly in real-time, to have access to themost up-to-date knowledge to be extremely time consuming or nearlyimpossible. For example, if a clinician is interested in how a medicalcolleague treats patients with similar disease manifestations, theclinician would have to manually browse through the electronic medicalrecords (EMR) for relevant medical reports or personally contact otherclinicians to update the clinician's knowledge. In another example, tobe acquainted with the most up-to-date knowledge outside the EMR,clinicians would have to manually sift through large volumes ofavailable information sources to extract the relevant content ofinterest. This process is tedious and prone to errors because importantinformation is missed by the clinicians leading to increased risk formedical errors and compromised patient safety.

Furthermore, the manual approach to searching information also leads tolong time delays in retrieving clinically relevant information which mayadversely affect efficient delivery of high quality care to patients.For example, if there is a sudden emergence of an infectious disease ora recent discovery of a critical clinical practice method, a clinicianwants to have this information to optimize any clinical decision makingprocess.

SUMMARY

The exemplary embodiments are related to a method for determining andnotifying a clinician of information relevant to the clinician. Themethod includes identifying at least one keyword in a user profile of aclinician, identifying at least one content word in a new informationitem, determining a relevance score between the new information item andthe clinician based on the at least one keyword and the at least onecontent word and when the relevance score is above a predeterminedthreshold value, generating a notification for the clinician indicatingthe new information item.

The exemplary embodiments are also related to a relevance server thathas a transceiver communicating via a communications network, thetransceiver configured to receive clinician information and a newinformation item and a memory storing an executable program. Therelevance server also has a processor that executes the executableprogram which causes the processor to perform operations includingidentifying at least one keyword in a user profile of a clinician,identifying at least one content word in a new information item,determining a relevance score between the new information item and theclinician based on the at least one keyword and the at least one contentword, and when the relevance score is above a predetermined thresholdvalue, generating a notification for the clinician indicating the newinformation item.

The exemplary embodiments are also related to a further method fordetermining and notifying a clinician of information relevant to theclinician. The further method includes receiving clinician informationassociated with a clinician, analyzing the clinician information toidentify at least one keyword to generate a user profile for theclinician, monitoring information sources for a new information item,when the new information item is detected, analyzing the new informationitem to identify at least one content word in the new information item,determining a relevance score between the new information item and theclinician based on the at least one keyword and the at least one contentword and generating a notification for the clinician indicating the newinformation item and a relevance factor of the new information itembased on the relevance score.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a system according to the exemplary embodiments.

FIG. 2 shows a relevance server of FIG. 1 according to the exemplaryembodiments.

FIG. 3 shows a method for determining relevant new available informationaccording to the exemplary embodiments.

DETAILED DESCRIPTION

The exemplary embodiments may be further understood with reference tothe following description and the related appended drawings, whereinlike elements are provided with the same reference numerals. Theexemplary embodiments are related to a device, a system, and a methodfor clinicians to provide a more efficient manner of care to patients bydetermining new available information that is relevant to a clinician.The exemplary embodiments are configured to automatically monitorinformation sources in real-time that are filtered to determine the mostrelevant content for a select clinician such that the clinician isnotified of relevant content to make better-informed clinical decisionsand improve healthcare quality and patient outcomes.

The exemplary embodiments minimize a time and an effort required by aclinician to manually review large volumes of clinical and biomedicalinformation sources for specific clinical information/knowledge thatpotentially enhances clinical acumen and ensure better outcomes fortheir patients. Accordingly, the exemplary embodiments are configured toautomatically monitor the available information streams in real-time,filter the most relevant content based on the interest profiles of theclinician, and deliver the content to the clinician promptly in aseamless and workflow-centric manner.

FIG. 1 shows a system 100 according to the exemplary embodiments. Thesystem 100 relates to a communication between various componentsinvolved in providing healthcare and/or health-related information to apatient or user from a clinician. Specifically, the system 100 mayinclude a plurality of information sources 105, 110, a communicationsnetwork 115, a clinician device 120, a profile repository 125, and arelevance server 130. As will be described in further detail below, thesystem 100 is configured to utilize the information sources 105, 110such that the healthcare and/or the health-related information may beprovided using the first and/or second mechanism according to theexemplary embodiments.

The information sources 105, 110 may represent any source from whichinformation is received. The information may be medical information,online or digital information, etc. For example, the information source105 may include a repository for clinical reports in an electronicmedical record (EMR). In another example, the information source 105 mayinclude other medical-related data from medical journals, hospitals,etc. In a further example, the information source 110 may include onlinestreams such as social media streams (e.g., a microblog website), healthblogs, online news media, etc. For exemplary purposes, the informationsources 105, 110 may provide any information that may be used inperforming the first and second mechanisms according to the exemplaryembodiments.

It should be noted that the system 100 illustrating two informationsources 105, 110 is only exemplary. The information sources 105, 110 mayrepresent one or more information sources that are configured to providethe information to the other components of the system 100. In fact, theinformation sources 105, 110 may represent each individual item that maybe available from a repository or source, the repository or sourceitself, a collection of repositories, etc.

The communications network 115 may be configured to communicativelyconnect the various components of the system 100 to exchange data. Thecommunications network 115 may represent any single or plurality ofnetworks used by the components of the system 100 to communicate withone another. For example, if the relevance server 130 is used at anadministration site, the communications network 115 may include aprivate network in which the relevance server 130 may initially connect(e.g. a hospital network). The private network may connect to a networkof an Internet Service Provider to connect to the Internet.Subsequently, through the Internet, a connection may be established toother electronic devices. It should be noted that the communicationsnetwork 115 and all networks that may be included therein may be anytype of network. For example, the communications network 110 may be alocal area network (LAN), a wide area network (WAN), a virtual LAN(VLAN), a WiFi network, a HotSpot, a cellular network (e.g., 3G, 4G,Long Term Evolution (LTE), etc.), a cloud network, a wired form of thesenetworks, a wireless form of these networks, a combined wired/wirelessform of these networks, etc.

The clinician device 120 may represent any electronic device that isconfigured to perform the functionalities associated with a clinician.For example, the clinician device 120 may be a portable device such as atablet, a laptop, etc. or a stationary device such as a desktopterminal. The clinician device 120 may include the necessary hardware,software, and/or firmware to perform the various operations associatedwith medical treatment. The clinician device 120 may also include therequired connectivity hardware, software, and firmware (e.g.,transceiver) to establish a connection with the communications network115 to further establish a connection with the other components of thesystem 100. For example, the clinician device 120 may scheduleappointments for patients using a calendar application, may tracktreatments or procedures of a patient, etc. In another example, theclinician device 120 may be used to post online content such asmicroblogs. In a further example and as will be described in furtherdetail below, the clinician device 120 may receive notifications fromthe relevance server 130 regarding new available information.

The profile repository 125 may be a component that stores user profiles.Specifically, the profile repository 125 may store user profiles ofclinicians. As will be described in further detail below, the relevanceserver 130 may generate user profiles that may be stored in the profilerepository 125. If the profile repository 125 already has a user profilefor a particular clinician, the relevance server 130 may query theprofile repository 125 to retrieve the corresponding user profile.

The relevance server 130 may be a component of the system 100 thatperforms functionalities associated with the first mechanism of theexemplary embodiments. FIG. 2 shows the relevance server 130 of FIG. 1according to the exemplary embodiments. The relevance server 130 mayprovide various functionalities in determining relevant new availableinformation and notifying a clinician of these relevant new availableinformation. Although the relevance server 130 is described as a networkcomponent (specifically a server), the relevance server 130 may beembodied in a variety of hardware components such as a portable device(e.g., a tablet, a smartphone, a laptop, etc.), a stationary device(e.g., a desktop terminal), incorporated into the personal device suchas a physician device, incorporated into a website service, etc. Therelevance server 139 may include a processor 205, a memory arrangement210, a display device 215, an input and output (I/O) device 220, atransceiver 225, and other components 230 (e.g., an imager, an audio I/Odevice, a battery, a data acquisition device, ports to electricallyconnect the reporting server 130 to other electronic devices, etc.).

The processor 205 may be configured to execute a plurality ofapplications of the relevance server 130. As will be described infurther detail below, the processor 205 may utilize a plurality ofengines including a profile engine 235, a monitoring engine 240, arefinement engine 245, a matching engine 250, and a notification engine255. The profile engine 235 may analyze interest profiles of a clinicianto determine an overall context of information needs through generationof a user profile of the clinician. The monitoring engine 240 maymonitor the information sources 105, 110 in real-time to track any newavailable information. The refinement engine 245 may process the resultsof the monitoring engine 240 to refine the results. The matching engine250 may determine select ones of the refined results based on the userprofile of the clinician. The notification engine 255 may generatenotifications for the clinician of the select ones of the refinedresults.

It should be noted that the above noted applications and engines eachbeing an application (e.g., a program) executed by the processor 205 isonly exemplary. The functionality associated with the applications mayalso be represented as components of one or more multifunctionalprograms, a separate incorporated component of the relevance server 130or may be a modular component coupled to the relevance server 130, e.g.,an integrated circuit with or without firmware.

The memory 210 may be a hardware component configured to store datarelated to operations performed by the relevance server 130.Specifically, the memory 210 may store data related to the variousengines 235-255 such as the user profile of the clinician and the datafrom the information sources 105, 110. The display device 215 may be ahardware component configured to show data to a user while the I/Odevice 220 may be a hardware component that enables the user to enterinputs. For example, an administrator of the relevance server 130 maymaintain and update the functionalities of the relevance server 130through user interfaces shown on the display device 215 with inputsentered with the I/O device 220. It should be noted that the displaydevice 215 and the I/O device 220 may be separate components orintegrated together such as a touchscreen. The transceiver 225 may be ahardware component configured to transmit and/or receive data via thecommunications network 110.

According to the exemplary embodiments, the relevance server 130 mayperform various different operations to determine which of the newlyavailable information is related to a particular clinician. Initially,as described above, the profile engine 235 may analyze interest profilesof a clinician to determine an overall context of information needsthrough generation of a user profile of the clinician. The profileengine 235 may provide an initial operation in which profile informationis gathered to determine the user profile of a particular clinician. Theprofile information related to the user profile that may be gathered maybe of any type and received from the information sources 105, 110 ormanually from the clinician. For example, the profile information mayinclude a resume of the clinician, expertise profiles saved in variousdepositories (e.g., a hospital management system), information collectedvia a short interactive online survey of the specific information needsof the clinician, etc.

The profile application 235 may be configured to analyze the profileinformation. For example, the profile application 235 may utilize atopic modelling/topic signature operation to extract topical keywordsthat capture the overall context of the interest profile of theclinician. The topic modelling operation may tend to discover abstracttopics from a collection of datasets through statistical analyses ofwords across documents. The topic signature operation may identify aword as a topical word if the word has significantly greater probabilityin a given text compared to a large background corpus. The extractedtopical keywords along with various possible n-gram combinations may beutilized to expand the topical vocabulary by extracting related synonymsets from an open source lexical database (e.g., WordNet) and viaexploiting deep neural word/phrase embeddings. The neural word/phraseembeddings may be trained from a large collection of data garnered fromvarious online information sources by using a deep learning-basedword/phrase to vector representation modelling operation. In thisframework, each word/phrase may be mapped to a unique vector using askip-gram model architecture. Once the training converges, words/phraseswith similar meanings may be co-located in the vector space such thatthe position and/or difference in positions may be exploited todetermine a relatedness among different words/phrases. The expandedtopical keyword list per user profile may be stored via a text indexingapplication for further analyses during real-time information contentfiltering, to be described in further detail below. In this manner, theprofile application 235 may generate a user profile of the clinicianincluding various keywords associated with the clinician.

The monitoring engine 240 may monitor the information sources 105, 119in real-time to track any new available information. The monitoringengine 240 may monitor various online and/or other information streamsincluded in the information sources 105, 110. For example, theinformation streams may include social media data such as Twitter, blogposts, online news media, EMR free text medical reports, etc. that aretracked in real-time to determine any incoming new posts and/or reports.

The refinement engine 245 may process the results of the monitoringengine 240 by refining the results. Specifically, the refinement engine245 may process incoming information, perform a tokenization operationand/or a parts-of-speech (POS) tagging operation, and remove noisyelements to generate a “clean” version of the new available informationidentified by the monitoring engine 240. It is noted that theinformation being monitored by the monitoring engine 240 may be streamedin various formats. For example, the information may be streamed asshort sentences (e.g., tweets from Twitter), as a collection of one ormore paragraphs (e.g., status posts from social media sites, blog posts,etc.), as documents and/or reports (e.g., from the EMR), etc. When newavailable information is received by the monitoring engine 240, therefinement engine 245 may process the new available information toremove all possible noise depending on the information source and basedon the format that the information is streamed. Specifically, noisycontent is removed using rule-based operations corresponding to thestreamed format in association with curated knowledge databases aboutdomain-dependent noisy words/templates. The refinement engine 245 mayapply natural language processing (NLP) operations such as tokenizationand POS tagging towards extracting important content words thatpreserves the contextual meaning of the content of the streamedinformation. In this manner, the clean version of the new availableinformation including content words of the new available information maybe generated.

The matching engine 250 may determine select ones of the refined resultsthat are appropriate for the clinician based on the user profile of theclinician. Specifically, the matching engine 250 may use NLP operationsand information retrieval techniques to analyze the clean version of thenew available information with respect to the user profile of theclinician to determine appropriate matches. The matching engine 250 mayutilize the content words from the refinement engine 245 as a query thatare matched with the keywords of the user profile of the clinician(e.g., by using a text indexing operation). The text indexing operationmay utilize the content words as various combinations of possiblen-grams/phrases to find an overall content match across the keywords ofthe user profile.

The relevance of an item in the new available information to the userprofile of the clinician may be measured in a two step process. In afirst step, the text indexing operation may return a termfrequency—inverse document frequency (TF-IDF) based content matchingscore with respect to the user profile of the clinician. Specifically, atext indexing score may be generated. In a second step, the keywordsfrom the user profile may be further utilized to determine a semanticsimilarity with the content words of the new available content through asemantic similarity measurement operation built on semantic networks ofrelated words and corpus-based statistics. Specifically, a semanticsimilarity score may be generated.

The matching engine 245 may utilize weighting factors that areassociated with the text indexing score and the semantic similarityscore. The weighting factors may provide a dynamic approach to utilizethe text indexing score and the semantic similarity score where agreater indexing or a greater similarity may allow the correspondingscore to be weighted accordingly and provide a more significant factor.Thus, the matching engine 245 may combine the text indexing scoreweighted by its weighting factor and the semantic similarity scoreweighted by its weighting factor to generate an overall relevance scoreof the new available information item to the user profile of theclinician. The overall relevance score may be indicative of how relevantthe new available information item is for a particular clinician. Thus,a first new available information item having a first overall relevancescore may be greater than a second overall relevance score of a secondnew available information item. Accordingly, the first new availableinformation item may have a higher relevance to the clinician. Thematching engine 245 may also determine a semantic similarity of a newavailable information item with other new available information itemsthat may have already been communicated to the clinician. This operationmay provide a redundancy check such that the clinician is notoverwhelmed by repeated information.

The notification engine 255 may generate notifications for the clinicianof the select ones of the refined results. Specifically, thenotification engine 255 may transmit content notifications to aclinician when new available information items are identified asrelevant (e.g., corresponding to the information of the user profile ofthe clinician). For example, if the overall relevance score of a newavailable information item is greater than a predetermined thresholdvalue (e.g., determined empirically after sufficient amount oftraining), the new available information item may be transmitted to theclinician. For example, the notification engine 255 may generate acorresponding message that is transmitted (e.g., mobile pushnotification in real-time, email, etc.). In this manner, the clinicianmay be aware of interesting new available information as soon as itbecomes available (e.g., within a few seconds of the new availableinformation becoming available).

The notification engine 250 may be configured to generate thenotifications in a variety of different manners. In a first example, ageneric message may be generated to indicate that new availableinformation items have been detected. In another example, a morespecific message may be generated that includes links or other pointersthat lead the clinician to the new available information item. In afurther example, a descriptive message may be generated in which a mostrelevant sentence or passage is included in the message regarding thenew available information item such that the clinician may read theprovided text and determine whether to proceed further. The descriptivemessage may also utilize the more specific message feature of includinglinks to easily allow the clinician to proceed.

The notification engine 250 may also be configured to be modified,particularly to limit the number of notifications over a period of time.In this manner, the clinician may not be inundated with all newavailable information items. In a first example, the number ofnotifications to be delivered may be controlled by setting an upperlimit (e.g., only 15 notifications per day). In a second example, thetime that notifications are delivered may be customized such as based onthe work schedule of the clinician so that notifications are notdelivered during an unwanted time period (e.g., when the clinician isdoing rounds, in surgery, or asleep). In a third example, the clinicianmay utilize a personalized predetermined threshold value so that morerelevant items (e.g., a higher value than the machine-learned value) orless relevant items (e.g., a lower value than the machine-learned value)may be included in the notification. In a fourth example, thenotifications may be bundled such that new available information itemsare stored and later analyzed such that the top new availableinformation items having at least a particular overall relevance scoreare included in a message to be delivered (e.g., email). The timing ofthis process may be an extended time period (e.g., once per day, onceper week, etc.).

In a specific implementation of the relevance server 130, results of newavailable information items may be determined based on various clinicalquestions to be satisfied. For example, a primary use case may relate toa scenario where a clinician may seek relevant research-based evidenceon how best to care for patients at the point of care. Specifically, theclinician may require specific information on the patient's most likelydiagnosis given a list of signs/symptoms, the most essentialtests/procedures in a given scenario, and the most effective treatmentplan given a diagnosis. Accordingly, the exemplary embodiments areconfigured with an objective of retrieving a ranked list of results thatanswer questions related to multiple categories of clinical informationneeds. In a particular example, short medical case reports areassociated with one of three generic clinical questions: “What is thepatient's diagnosis?”, “What tests should the patient receive?”, and“How should the patient be treated?”. The results may be judged in termsof their relevance to the corresponding clinical question. This mayrelate particularly to the monitoring engine 240 and the refinementengine 245.

As described above, the exemplary embodiments provide a NLP-drivenmethod that combines syntactic, semantic, and filtering operationstowards extracting relevant biomedical articles corresponding toclinical concepts (e.g., diagnoses, treatment, and/or tests) relevant toeach given topic. This particular implementation may also utilize theabove engines such as (i) the topical analysis of identifying the mostrelevant content words from the information sources 105, 110, (ii) theclinical inferencing where reasoning through the content words arrive atthe diagnoses, tests, and treatments based on underlying clinicalcontexts by using neural phrase embeddings and/or an external clinicalknowledge base, and (iii) relevant article retrieval where retrievingand ranking pertinent biomedical articles based on the content words andclinical inferences from (i) and (ii).

In the topical analysis, the TF-IDF described above may be utilized fromgiven descriptions and/or summaries and mapped to categories representedin controlled clinical vocabularies and/or ontologies. The TF-IDF mayalso be identified relevant to demographic information, interpretedvital patient parameters based on standard normal range values, andfiltered out negated clinical concepts to give more weight to positiveclinical manifestations in a given patient scenario. Those skilled inthe art will appreciate that the use of clinical domain ontologies maybe particularly effective as they have been implemented to promotestandard clinical vocabulary and are widely used to semanticallycategorize clinical concepts, and facilitate information exchange andinteroperability.

In the clinical inferencing, a word/phrase-to-vector neural phraseembedding model is used (which has been trained on over 4 millionclinically relevant sentences garnered from multiple clinical datasources, articles, and discharge summaries) to capture the overallcontext of a given topic description or summary towards inferring thedifferential diagnoses based on the commonest clinical diagnosesrepresented in clusters of identified topical content words from thetopical analysis step. A skip-gram model architecture may be utilized tolearn vector representations of words and phrases as reported to provideimproved results. The list of possible diagnoses may be furthervalidated, filtered, and ranked by referencing a clinical knowledge basewhich is indexed, and a list of candidate articles with relevantdiagnoses corresponding to each topical content word may be extracted.Through this process, relationships between topical content words andassociated clinical concepts (e.g., diagnoses/disorders, treatment, andtest) may be found within a comprehensive knowledge base for the purposeof biomedical evidence retrieval.

In the relevant article retrieval, topical content words and thecorresponding disorders/diagnoses, tests, and treatments obtained fromthe clinical inferencing operation may be used to retrieve candidatebiomedical articles by searching through abstracts of given articles.Candidate results may be ranked using multiple weighting operationsdesigned to address the three types of clinical questions (e.g.,diagnosis, test, and treatment). The retrieved results may be furtherfiltered by location, demographic information, and other contextualinformation from the topic description/summary towards improving arelevance of the results. The final list may further be ordered bypublication date of the new available information items to provide achronological biomedical evidence for the answers to each topic.

In performing the above implementation of the relevance server 130according to the exemplary embodiments, an experimental set of data maybe used. For example, a test dataset may include thirty topics dividedinto three question types such as those described above. The given topicdescriptions or topics may be essentially medical case narratives thatdescribe scenarios related to patient medical history, signs/symptoms,diagnoses, tests, and treatments. The topics may be provided in twoversions depending on the depth of information. Topic descriptions mayinclude comprehensive descriptions of the patient situation whereastopic summaries may contain the most important information. Furthermore,ground truth diagnoses may be provided for the test and treatmenttopics.

Running an experiment with the above datasets, an evaluation may beconducted using standard evaluation procedures (e.g., Text RetrievalConference (TREC) procedures) for ad hoc information retrieval tasks.The highest ranked biomedical articles may be sampled and judged bymedical domain experts on a three-point scale of 0 (not relevant), 1(possibly relevant), and 2 (definitely relevant) depending on therelevance of the answer to the associated question type about a givenscenario. The results indicate that the clinical question answeringsystem according to this implementation of the exemplary embodimentsperform close to median scores for all evaluation measures. Analysis ofthe results also demonstrate that the clinical question answering systemaccording to the exemplary embodiments may achieve better results forcertain topics when topic summaries are used whereas neural word/phraseembeddings improve upon scores for a number of topics. The results alsoindicate that the identification and use of accurate differentialdiagnoses has a significant impact on the accuracy of the relevantbiomedical article retrieval.

This implementation of the relevance server 130 according to theexemplary embodiments may also utilize a real-time filtering system,particularly of a microblog. Using the above engines, this particularimplementation may (i) analyze user profiles that leverage neuralword/phrase embeddings for contextual understanding, (ii) analyzemicroblog content where a noisy element filtering operation as well as atokenization and POS tagging operation for generation of a cleanedversion of the microblog are performed, and (iii) matching relevantcontent where mapping of relevant microblogs to a corresponding userprofile is through a combination of a TF-IDF based content matchingscore and a semantic similarity score. This may relate particularly tothe profile engine 235 and the matching engine 250.

In the analysis of user profiles, a plurality of different user profilesmay be analyzed using a topic signature operation that extracts the mostimportant keywords to capture the overall context of the informationneed. The keywords along with a corresponding n-gram combination may beutilized to expand the topical vocabulary by extracting related synonymsets and exploiting deep neural word/phrase embeddings. The neuralword/phrase embeddings may be trained from over sixty million microblogsby using a deep learning-based word/phrase to vector representationmodeling operation. The expanded keyword list per user profile may beindexed for further analyses during real-time microblog contentfiltering.

In the microblog content analysis, each incoming microblog may beprocessed to remove all noise using various rule-based operations inassociation with curated databases of known noisy elements that arewidely used in microblogs. The tokenization and POS tagging may beapplied to extract the most important words that preserve the contextualmeaning of the microblog.

In the relevant content matching, the content words in the abovedescribed clinical question answering system and the keywords may beused as a query for which an appropriate user profile is retrieved andmatched. The query may be transformed as various combinations ofpossible n-grams/phrases to determine an overall content match acrossthe user profile. The final relevance therebetween may be measured usinga weighted combination of two scores, the TF-IDF based content matchingscore and the semantic similarity score based on an operation built onsemantic networks of related words and corpus-based statistics.Subsequently, the notification engine 255 may be used in notifying anymatches.

FIG. 3 shows a method 300 for determining relevant new availableinformation according to the exemplary embodiments. Specifically, themethod 300 may relate to the mechanism of the exemplary embodiments inwhich a user profile is used to identify select ones of new availableinformation items that are to be identified to a clinician associatedwith the user profile. Accordingly, the method 300 will be describedfrom the perspective of the relevance server 130. The method 300 willalso be described with regard to the system 100 of FIG. 1 and theplurality of engines 235-255 of the relevance server 130 of FIG. 2.

In step 305, the relevance server 130 via the profile engine 235 mayreceive information associated with a clinician. As described above, theinformation associated with the clinician may be received from theinformation sources 105, 110. For example, documentation of theclinician may be received such as a resume, an expertise profile, etc.In another example, the relevance server 130 may receive online datasuch as microblog information of the clinician. In step 310, therelevance server 130 via the profile engine 235 may generate a userprofile for the clinician. Specifically, the information of theclinician may be analyzed to determine keywords that are included in theuser profile.

In step 315, the relevance server 130 may determine new availableinformation items. Specifically, the relevance server 130 via themonitoring engine 240 may monitor the information sources 105, 110 suchas social media, blog posts, online news media, EMR free text medicalreports, etc. The information sources 105, 110 may be updatedperiodically or dynamically by authors who add or post new availableinformation. The relevance server 130 may be configured to identify thenew available information items from a previous time.

In step 320, the relevance server 130 via the refinement engine 245 mayrefine the new available information. Specifically, the relevance server130 may perform a tokenization operation and/or a POS tagging operationto remove noisy elements in the new available information items. Thenoisy elements may relate to portions of the new available informationthat is irrelevant to the features of the exemplary embodiments. Therelevance server 130 may accordingly generate content words based on aclean version of the new available information items.

In step 325, the relevance server 130 via the matching engine 250 maymatch the content words of the new available information items with thekeywords of the user profile of the clinician. Specifically, using NLPoperations and information retrieval techniques, select ones of the newavailable information items may be identified as being relevant to theuser profile of the clinician. Using a text indexing score with acorresponding weight and a semantic similarity score with acorresponding weight, an overall relevance score may be determined whichindicates the relevance of a new available information item is to theuser profile of the clinician. It is noted that new availableinformation items that have no relevance may have a zero value whereasnew available information items that have at least some relevance mayhave a positive value.

In step 330, the relevance engine 130 via the notification engine 255may generate a notification for the clinician regarding any newavailable information items that have at least some relevance (positiveoverall relevance score). It is noted that the notification engine 255may also utilize a predetermined threshold value as the basis of whethera new available information item is to be used in the notification(e.g., only if the overall relevance score of the new availableinformation item is greater than the predetermined threshold value). Thenotification may be transmitted in a variety of different forms, at avariety of different times, using a variety of different factors, etc.The relevance engine 130 may also report any new available informationitem and include the overall relevance score to indicate how the newavailable information item relates to the user profile of the clinician.

The exemplary embodiments described above relate to clinicians and amedical field in which medical information is identified for theclinician. However, the use of the medical-related implementation isonly exemplary. Those skilled in the art will understand that theexemplary embodiments may be modified accordingly to be used with anyuser profile and any document retrieval system based on the userprofile.

The exemplary embodiments provide a device, system, and method ofdetermining information relevant to a clinician. The exemplaryembodiments provide a profile mechanism in which a user profile of aclinician is determined through a plurality of keywords. The exemplaryembodiments provide an information mechanism in which new availableinformation is identified and content words associated therewith aredetermined. The exemplary embodiments provide a matching mechanism inwhich a relevance of the new available information items are determinedfor the user profile of the clinician such that relevant new availableinformation items are notified to the clinician.

Those skilled in the art will understand that the above-describedexemplary embodiments may be implemented in any suitable software orhardware configuration or combination thereof. An exemplary hardwareplatform for implementing the exemplary embodiments may include, forexample, an Intel x86 based platform with compatible operating system, aWindows platform, a Mac platform and MAC OS, a mobile device having anoperating system such as iOS, Android, etc. In a further example, theexemplary embodiments of the above described method may be embodied as acomputer program product containing lines of code stored on a computerreadable storage medium that may be executed on a processor ormicroprocessor. The storage medium may be, for example, a local orremote data repository compatible or formatted for use with the abovenoted operating systems using any storage operation.

It will be apparent to those skilled in the art that variousmodifications may be made in the present disclosure, without departingfrom the spirit or the scope of the disclosure. Thus, it is intendedthat the present disclosure cover modifications and variations of thisdisclosure provided they come within the scope of the appended claimsand their equivalent.

1. A method, comprising: at a relevance server: identifying at least onekeyword in a user profile of a clinician; in real time, monitoringinformation sources for a new information item; identifying at least onecontent word in a new information item; determining a relevance scorebetween the new information item and the clinician based on the at leastone keyword and the at least one content word; and when the relevancescore is above a predetermined threshold value, generating anotification for the clinician indicating the new information item. 2.The method of claim 1, further comprising: receiving clinicianinformation; determining the at least one keyword in the clinicianinformation; and generating the user profile by including the at leastone keyword.
 3. The method of claim 2, wherein the clinician informationis based from at least one of a resume, available expertise profiles,surveys, and online content.
 4. The method of claim 1, furthercomprising: monitoring information sources for the new information item;and determining at least one first content word in the new informationitem, the at least one content word included in the at least one firstcontent word.
 5. The method of claim 4, wherein the information sourcesis based on at least one of social media data, blog posts, online newsmedia, and electronic media records (EMR) reports.
 6. The method ofclaim 4, wherein the at least one first content word is refined byremoving noisy content therein.
 7. The method of claim 1, wherein therelevance score includes a text indexing score and a semantic similarityscore.
 8. The method of claim 7, wherein the text indexing score isbased on a term frequency-inverse document frequency (RF-IDF) contentmatching operation, and wherein the semantic similarity score is basedon semantic networks of related words and corpus-based statistics. 9.The method of claim 7, wherein the text indexing score is applied with afirst weighting factor and the semantic similarity score is applied witha second weighting factor.
 10. The method of claim 1, furthercomprising: transmitting the notification to the clinician based on atleast one of a total number of notifications within a predetermined timeperiod, a time of day, and a collection preference.
 11. A relevanceserver, comprising: a transceiver communicating via a communicationsnetwork, the transceiver configured to receive clinician information anda new information item; a memory storing an executable program; and aprocessor that executes the executable program that causes the processorto perform operations, comprising, identifying at least one keyword in auser profile of a clinician, in real time, monitoring informationsources for a new information item; identifying at least one contentword in a new information item, determining a relevance score betweenthe new information item and the clinician based on the at least onekeyword and the at least one content word, and when the relevance scoreis above a predetermined threshold value, generating a notification forthe clinician indicating the new information item.
 12. The relevanceserver of claim 11, wherein the operations further comprise determiningthe at least one keyword in the clinician information and generating theuser profile by including the at least one keyword.
 13. The relevanceserver of claim 12, wherein the clinician information is based from atleast one of a resume, available expertise profiles, surveys, and onlinecontent.
 14. The relevance server of claim 11, wherein the operationsfurther comprise monitoring information sources for the new informationitem and determining at least one first content word in the newinformation item, the at least one content word included in the at leastone first content word.
 15. The relevance server of claim 14, whereinthe information sources are based on at least one of social media data,blog posts, online news media, and electronic media records (EMR)reports.
 16. The relevance server of claim 14, wherein the at least onefirst content word is refined by removing noisy content therein.
 17. Therelevance server of claim 11, wherein the relevance score includes atext indexing score and a semantic similarity score.
 18. The relevanceserver of claim 17, wherein the text indexing score is based on a termfrequency-inverse document frequency (RF-IDF) content matchingoperation, and wherein the semantic similarity score is based onsemantic networks of related words and corpus-based statistics.
 19. Therelevance server of claim 17, wherein the text indexing score is appliedwith a first weighting factor and the semantic similarity score isapplied with a second weighting factor.
 20. A method, comprising: at arelevance server: receiving clinician information associated with aclinician; analyzing the clinician information to identify at least onekeyword to generate a user profile for the clinician; monitoring, inreal time, information sources for a new information item; when the newinformation item is detected, analyzing the new information item toidentify at least one content word in the new information item;determining a relevance score between the new information item and theclinician based on the at least one keyword and the at least one contentword; and generating a notification for the clinician indicating the newinformation item and a relevance factor of the new information itembased on the relevance score.